Refactor AutoIncrementTracker into generitc type with interfaces by nicktobey · Pull Request #11337 · dolthub/dolt

nicktobey · 2026-07-22T06:15:57Z

The purpose of this PR is to enable Doltgres to create a global state for Sequences that behaves similarly to Dolt's global state for auto increment columns. In order to allow for code reuse, we have to do the following:

Create a generic type containing the behavior common to both auto-increment and sequences.
Create new interfaces that can be implemented to describe the behavior unique to each feature.

(Bikeshedding note: I use the term "Relation" to describe a top-level object in the database, and encompasses both tables and root objects. I'm open to alternative names.)

The new interfaces can be found in go/libraries/doltcore/sqle/globalstate/sequences/state.go

SequenceState is an interface for a state machine that can be incremented. Each time it is incremented, it produces a new SQL value. SequenceStates can also be compared to each other to determine which one is "further along" and "merged" to create a single state that is further along in its sequence than any of the input states. The only implementation in Dolt is AutoIncrementState, which is just a uint64. Doltgres sequences will have a more complicated implementation, since they have more parameters that affect how the state is incremented.
SequencedRelation is a relation that holds a SequenceState. The only implementation in Dolt is doltdb.Table, which can hold an auto-increment value. Doltgres Sequences will also implement this interface, and possibly Doltgres tables with a SERIAL column.

The common logic is in the SequenceTracker type, which has an interface in go/libraries/doltcore/sqle/globalstate/sequence_tracker.go and an implementation in go/libraries/doltcore/sqle/dsess/sequence_tracker.go

AutoIncrementTracker is now a type alias for a specialization of SequenceTracker.

Finally, while GlobalState previously only held an AutoIncrementTracker, it now contains a map from arbitrary keys to SequenceTrackers. This will allow Doltgres to add an additional SequenceTracker for global tracking of Doltgres sequences.

There are a couple places in this PR that are TODOs, because they are corner cases that aren't reachable for AutoIncrement but are reachable for Doltgres sequences. They mostly concern how to handle when branches contain incompatible Sequences. I haven't decided exactly how this should be handled, and this PR will not be merged until we know what to do in that case, but it shouldn't affect the review of the rest of the PR.

…te.sh

coffeegoddd · 2026-07-22T06:39:55Z

@nicktobey DOLT

read_tests	from_latency	to_latency	percent_change
covering_index_scan	2.3	2.3	0.0
groupby_scan	142.39	142.39	0.0
index_join	1.93	1.93	0.0
index_join_scan	1.32	1.32	0.0
index_scan	219.36	215.44	-1.79
oltp_point_select	0.25	0.25	0.0
oltp_read_only	5.0	5.0	0.0
select_random_points	0.51	0.51	0.0
select_random_ranges	0.64	0.64	0.0
table_scan	200.47	207.82	3.67
types_table_scan	458.96	475.79	3.67

write_tests	from_latency	to_latency	percent_change
oltp_delete_insert	6.09	6.09	0.0
oltp_insert	3.13	3.13	0.0
oltp_read_write	11.24	11.24	0.0
oltp_update_index	3.25	3.3	1.54
oltp_update_non_index	2.97	2.97	0.0
oltp_write_only	6.21	6.21	0.0
types_delete_insert	6.79	6.79	0.0

coffeegoddd · 2026-07-22T06:48:08Z

@nicktobey DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
`3fc7f98`	ok	5937471

version	total_tests
`3fc7f98`	5937471

correctness_percentage
100.0

coffeegoddd · 2026-07-22T06:51:06Z

@nicktobey DOLT

test_name	from_latency_p95	to_latency_p95	percent_change
tpcc-scale-factor-1	45.79	45.79	0.0

test_name	from_server_name	from_server_version	from_tps	to_server_name	to_server_version	to_tps	percent_change
tpcc-scale-factor-1	dolt	`a0b1bf5`	52.71	dolt	`97264f4`	53.05	0.65

coffeegoddd · 2026-07-22T06:55:21Z

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
`d419eb8`	ok	5937471

version	total_tests
`d419eb8`	5937471

correctness_percentage
100.0

coffeegoddd · 2026-07-29T00:43:41Z

@nicktobey DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
`ddaf2b8`	ok	5937471

version	total_tests
`ddaf2b8`	5937471

correctness_percentage
100.0

Hydrocharged

LGTM! Just some minor nitpicks

Hydrocharged · 2026-07-29T13:21:19Z

-		mm:         mutexmap.NewMutexMap(),
-		init:       make(chan struct{}),
-		cancelInit: make(chan struct{}),
+func NewSequenceTrackerFromRoots[


In general, I think it would be worthwhile to add function and struct comments, in case someone else needs to modify these in the future.

Hydrocharged · 2026-07-29T13:37:12Z

-
-			if seq > newHighestValue {
-				newHighestValue = seq
+			if newHighestValue == nil {


It's a nitpick, but technically for Doltgres, negative sequences aren't the highest values, so I'd give this a different name.

zachmu

Overall looks good, one pretty annoying comment about using doltdb.TableName in these interface.

zachmu · 2026-07-29T20:24:00Z

+	DropTable(ctx *sql.Context, relation string, wses ...*doltdb.WorkingSet) error
+	// InitWithRoots fills the SequenceTracker with values pulled from each root in order.
+	InitWithRoots(ctx context.Context, roots ...doltdb.Rootish) error
+	Close()


Should document the contract and expectations for close

zachmu · 2026-07-29T20:24:44Z

+)
+
+// SequenceTrackerBase is the non-generic base interface for SequenceTracker
+// It is useful for establishing an upper bound on type parameters in some circumstances.


Is this the best way to do this? Don't want you to rat-hole too deep but this is a little odd.

zachmu · 2026-07-29T20:46:42Z

+	StateType sequences.SequenceState[StateType, ValueType],
+	ValueType comparable,
+] interface {
+	GetRelation(ctx context.Context, root doltdb.RootValue, tName doltdb.TableName) (relation RelationType, resolvedName string, found bool, err error)


Method doc plz

coffeegoddd · 2026-07-30T22:43:49Z

@nicktobey DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
`ef151f6`	ok	5937471

version	total_tests
`ef151f6`	5937471

correctness_percentage
100.0

…ameters.

…rn an iter.Seq

coffeegoddd · 2026-07-31T01:07:17Z

@nicktobey DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
`97264f4`	ok	5937471

version	total_tests
`97264f4`	5937471

correctness_percentage
100.0

nicktobey and others added 5 commits July 16, 2026 14:09

Allow GlobalState to have multiple AutoIncrementTrackers

a8eba90

Add type-safe synchronized Map implementation.

debe9fe

Add globalstate/sequences package of interfaces

754c6bf

Have AutoIncrementTracker implement SequenceTracker

3fc7f98

[ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/upda…

d419eb8

…te.sh

coffeegoddd added the performance_approved label Jul 22, 2026

coffeegoddd added the correctness_approved label Jul 22, 2026

nicktobey added 2 commits July 28, 2026 17:12

Rename function to NewSequenceTracker

bcb1aaf

Rename and refactor code for managing global state.

ddaf2b8

Fix returning loaded sequence state.

2eee36f

Hydrocharged approved these changes Jul 29, 2026

View reviewed changes

nicktobey assigned nicktobey and zachmu and unassigned nicktobey Jul 29, 2026

zachmu approved these changes Jul 29, 2026

View reviewed changes

Rename types and add documentation based on PR feedback.

ef151f6

nicktobey added 2 commits July 30, 2026 15:45

Change SequenceTracker to take doltdb.TableName instead of string par…

4f7465e

…ameters.

Rename SequenceTracker.GetRelations to IterRelations and have it retu…

97264f4

…rn an iter.Seq

Uh oh!

Uh oh!

Conversation

nicktobey commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coffeegoddd commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coffeegoddd commented Jul 22, 2026

Uh oh!

coffeegoddd commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coffeegoddd commented Jul 22, 2026

Uh oh!

coffeegoddd commented Jul 29, 2026

Uh oh!

Hydrocharged left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Hydrocharged Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Hydrocharged Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

zachmu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

zachmu Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

zachmu Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zachmu Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coffeegoddd commented Jul 30, 2026

Uh oh!

coffeegoddd commented Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nicktobey commented Jul 22, 2026 •

edited

Loading

coffeegoddd commented Jul 22, 2026 •

edited

Loading

coffeegoddd commented Jul 22, 2026 •

edited

Loading